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Tail asymptotic of the stationary distribution for the state 
dependent (l,R)-refiecting random walk: near critica^ 
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In this paper, we consider the state-dependent reflecting random 

walk (RW) on the half line, allowing the size of jumps to the right at 
maximal R and to the left only 1. We provide an explicit criterion for 
positive recurrence and the explicit expression of the stationary distri- 
bution based on the intrinsic branching structure within the walk. As an 
application, we obtain the tail asymptotic for the stationary distribution 
in the "near critical" situation. 
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Introduction and Main Results 



L_i 1.1 The background and motivation 



We consider the (1, i?)-reflecting random walk on the half line, i.e., a Markov chain {Xm}m>Q 
on Z"*" = {0, 1,2, . . .} with Xq = and the transition probabilities Pij specified by for i > 
(^(0) = 0), 



r{i), for j = i, 

q(i), for j = i-l, 

Pj-i{i), iox i < 3 < i + R, 

0, otherwise, 



where r{i) + q{i) + pi{i) + P2{i) + ••• + = 1, < q{i) < 1, for i > 1, and r{i) > 

0!Pi(0)P2(0; ■ ■ ■ )P-r(^) — 0- Obviously, this Markov chain is irreducible. It can also be written 
as the transition matrix (for simplicity, R = 2), 



( ^0) 
5(1) 



v 



m(o) 

r(l) 
9(2) 



P2(0) 

Pi(l) 
r(2) 

9(3) 



P2(l) 
Pi (2) 
r(3) 



P2(2) 

Pi (3) P2(3) 
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in which ah unspecified entries are zero. 

For simphcity, we wiU restrict ourselves to consider R = 2, and we write the transition 
probabihty at position i as P{i) = {q{i),r{i),pi{i),p2{i)) (recall q{0) = and < q{i) < 1). At 
first, if the transition probability of the (1,2)-RW {X„i)m>o is state independent, i.e., P{i) = 
P = {q,r,pi,p2) for i > 1. Let (see figure 1) 

D = {iq,r,pi,p2) : PI+P2 + q + r = I; pi+2p2 < q}, 
L = {iq,r,pi,p2) : pi + P2 + q + r = 1; pi + 2p2 = q}, 

it is easy to see that {Xm)m>o is positive recurrent iff P{i) = P = {q,r,pi,p2) € D {i > I) and 
null recurrent iff P{i) = P = {q,r,pi,p2) G L (i > 1). 

How about the situation for the state- dependent (1, ii)-RW {Xm)m>o ? To our best knowledge 
only for R = 1, i.e., state- dependent (1, 1)-RW, the criteria for the (positive) recurrence and the 
expression for the stationary distribution have been given explicitly (see for example [12] and 
|13j). and further tail asymptotic for the stationary distribution have been found in ^ ( Page 
294 and Page 305). 

The aim of the present paper is to give an explicit criteria of the positive recurrence and ex- 
plicit expressions of the stationary distribution for the state- dependent (1, i?)-RW, which enable 
us to consider the tail asymptotic of the stationary distribution. Our method is based on the 
intrinsic branching structure within the random walk (jH], [S]). 



1.2 Main results 

1.2.1 Criteria for the positive recurrence and stationary distribution 
Define 

ak=PkiO)+Pk+i{0) + ---+PB{0), for l<k<R, 
a = {ai,a2, ■ ■ ■ ^ur), ei = (1,0, •• • ,0), 

Pk{i) +Pk+i{i) H VpR-i{i) +PR{i) 



Ok{i) 



Mi 



( Oi{i) 
1 





V 

Theorem 1.1. Assume for i > 0, 



m 





dR.lii) dRii) \ 
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:i.i) 



RxR 



/x(O) 



1 

W) 



aMiM2---Mi-ie'^. 



:i.2) 



(i) /i(i) (i > 0) are the stationary measure of the state- dependent {1, R)-RW {Xm)m>o- 

(ii) -(f Si^o /^(^) ^ then the walk {Xm}m>o is positive recurrence. Furthermore the 
stationary distribution can be expressed as 

li{i) 



TT{i) 



[1.3) 
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□ 



Remark ( 1.2 ) generalize the classical results for the state- dependent (1, 1) 
m (Page 297). 



■RW, see for example 

□ 



1.2.2 Tail asymptotic of the stationary distribution: near critical 



With the explicit expression of the stationary distribution (1.3) at hand, we can consider the 
tail asymptotic of the distribution. Firstly, it is not difficult (but is also not obviously, as [10] for 
the (L, 1)-RW) to see that the tail of 7r(i) is geometric decay in the sense limj_>oo = — c < 

when the transition probability P{i) — )• P = {q,r,pi,p2) £ D. What we are now interested in 
is the "near critical" situation: the transition probability P{i) from the interior of the "positive 
recurrence area D" to P in the "null recurrence area L" as z — )• oo. See figure 1 (In this figure, 
we assume r=0). 

q 




figure 1: District of the transition probability 

One of the interesting phenomena is that even all the P{i) € D, the "positive recurrence area", 
the walk could be null recurrent. To this end, we need to consider a finer manner of the 
P{i) goes to P as z — )• oo. Let P = {pi,P2,r, q) G L, and for i > 1, P{i) = {q{i) , r{i) , pi{i) , p2{i)) 
is given by 

Pi{i) =Pi- Si, P2{i) =P2- Si, q{i) = q + eu 
r{i) = l- pi{i)- p2{i) - q{i). 

where > and small enough, | as i ^ oo. It is obvious that P{i) & D, P £ L, and 
P{i) — )• P as i — )• oo. 

Theorem 1.2. (a) IfYli^o^i ^ recurrence, 
(h) ifY^'i^o^^ < ^ f^''' some l<p<2, k = |. 

(hi) When X^i^o 111=0 < -^"^ positive recurrence, and 

i 

log7r(i) ~ — K^^efc, as i ^ oo. 

k=0 
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(b2) When Yli^oYYk=o^ '^^^ ~ -^"m- '^^^^ recurrence. □ 
As an application, we immediately have the following 

Corollary 1.1. Suppose Si ~ Ci~" as i ^ oo, C is a positive constant. 
Case 1 : a > 1, Xm is null recurrence. 

Case 2 : ^ < a < 1, Xm is positive recurrence, and we have 

lognn) ~ 1 , as I ^ oo. 

1 — a 

Case 3 : a = 1, If Ck < 1, Xm is null recurrence; if Ck > 1, Xm is positive recurrence, and 

log7r(z) ~ —Cnlogi, as i ^ oo. 

□ 



Remark Theorem 1.2 and Corollary 1 1 . 1 1 say that even all the P{i) G D, the walk Xm could be 



null recurrent, which generalize the results for the state- dependent (1, 1)-RW ([4j, Page 294 and 
Page 305). □ 



We arrange the remainder of this paper as follows. In Section [2j we will prove Theorem 1.1 



after a brief review about the intrinsic branching structure within the walk, which is the basic 
tool to specify the stationary measure; Theorem 1.2 and Corollary 1 1 . 1 1 will be proved in Section 



3.1, together with some preparations on the asymptotic solution of difference system. 



Proof of Theorem 1.1 



Let T = inf{m > : Xm = 0}, N{i) = X]m=o -'-{Xm=i} the number of visits to state i by 
the chain before T, and is the expectation when the walk starts at Xq = i. Firstly, recall a 
classical results on the (positive) recurrence and the stationary distribution of a general Markov 
chain Xm- 

Proposition 2.1. (Thoerem (4.3), gj/j For k>0, /i(f) = E'^N{i) defi nes a stationary measure. 
If X^^o /^(^) ^ random walk is positive recurrence, and the stationary distribution can 

he expressed as 

□ 

We can calculate the /i(i) = E^N{i) by the intrinsic branching structure within the (l,i?)- 
RW as follows (the proof will delay at the end of this section) , 

Proposition 2.2. We have S°A^(0) = 1, and 

q{t) q{l) 
E°N(i) = -^aMiM2 ■ ■ ■ Mi.ie'i, for i > 1, 
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where a, Mi are given in (1.1). □ 



Proof of Theorem \l.l\ With Proposition 2.1 and 2.1 at hand, Theorem 1.1 is immediately. □ 



What we should do is to prove Proposition |2.2[ our method is the intrinsic branching structure 
within the (l,i?)-RW ([8], 2009). 

Brief review for the intrinsic branching structure. The intrinsic branching structure within a 
random walk has been studied by many authors. For the (1, 1)-RW, Dwass ([5], 1975) and Kesten 
et al. (P^, 1975) observed a Galton- Watson process with the geometric offspring distribution 
hidden in the nearest random walk. The branching structure is a powerful tool in the study of 
random walks in a random environment (RWRE, for short). In [TJ], Kesten et al.., proved a 
stable law for the nearest RWRE by using this branching structure. The key point is that the 
hitting time Tj can be calculated accurately by the branching structure. 

However, if the random walk is allowed to jump even to a bounded range, referred to as the 
(-L, i?)-RW, the situation will become much more complicated. A multi-type branching process 
has been revealed by Hong & Wang ([8], 2009) for the (L, 1)-RW, and a little bit late for the 
(l,i2)-RW ([9], 2010) by Hong &: Zhang. It must be emphasized that these two branching 
structures are not symmetric, instead they are essentially different. Note that if we assume 
q2{i) = 0, both branching structures degenerate to the case of the (1, 1)-RW. 

The following discussion is based on i? = 2. The general case can be similarly discussed, 
but the notation is much more complicated. Assume that Xq = we can calculate E^Ni by 
using the branching structure within the random walk ([8j, 2009). Note that we consider the 
reflected (l,i?)-RW and calculate E^Ni before first return the start position 0, actually we 
use the branching structure for the {R, 1)-RW by Hong & Wang ([8], 2009), and with a little 
modification because of considering the walk could be stay at each state i (here r{i) > 0). 

Recall that T = inf{n > 0, X„ = 0}, define 

U]^ = #{0 < j <T : Xj < k, Xj+i = k + m} for /c > 0, m = 1, 2. 
[/| = #{0 < j < T : Xj = k, Xj+i = k} for k>0, 

Setting 

Uk = {Ululul) for A:>0. 
We then have the following property (with a little modification) 

Theorem A (Hong and Wang |B]) (1) The process {Un}'^=Q is a 3-type branching process whose 
branching mechanism is given by, 



(2.11 





= (1,0,0)) 


= Pi(0) 


P{Uo 


= (0,1,0)) 


= P2(0) 


P{Uo 


= (0,0,1)) 


= r(0); 



and for A; > 



P{Uk+i = ia,b,c)\Uk = ei) = ^-^^±t^r{krp^ik)%{krqik), 
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P{Uk+i = (a,l + 6,c)|C/fc = 62) = 
P([/fc+i = (0,0,0)|C/fc = e3) = l. 



alblcl 



(2) For the process {[/„}^q, let he the 3x3 mean matrix whose m-th row is E{Uk+i\Uk 
Cm), for k >0. Then, one has that 



Mk 



/ Pi(fc) P2{k) rjk) 

I q{k) q(k) q(k 

1 _|_ Pi(fc) P2(fc) rjk 

q{k) q{k) q{k) 



k> I. 



V 











□ 



Now we are at the position to prove Proposition 2.2 



Proof of Proposition 2.2 It is not hard to deduce the relationship between the random walk 

and the intrinsic branching structure that E^N{0) = 1, and for i > 1, N{i) = Ul_^ + 1^*1 
(where|C/i| = [// + C/2 + c/3). 



^°iV(l) = pi(0) + S°?7oMi(l, 1, 1)' 



■ pi{0) + E°Uo{ 



1, 



1 



,0)' 



'9(1) 9(1) 
= pi(0) +pi(0)(^ _ 1) +p2(o) 

^ Pl(0)+P2(0) 

9(1) ■ 

For i > 1, using the Markov property, we have 

EO(Ar(i)|c/._,,t/,_2,...,C/o) = ul^ + \a,.iMi\. 

As a consequence 

E^Nii) = S°C/i_ie; + ^°[/,_iM,(l, 1, 1)' 

= E^Ui-2Mi^ie[ + S°t/,_2Mi-iMi(l, 1, 1)' 

= E^UoMiM2 ■ ■ ■ Mi_ie[ + E^UqUA ■ ■ ■ Mi-iMi{l, 1, 1)'. 

By ((2l]), E^Uo = (pi(0),p2(0),r(0)) := /3, 

E^Nii) = PM1M2 ■ ■ ■ Mi-ie[ + /3M1M2 • • • Mi-iMi{l, 1, 1)' 



Qii) 



/3MiM2---M,_i(l,l,0)'. 



(2.2) 



Define 



/ Pi{i)+P2{i) P2{i) r(i) 
q(i) q{i) q(i) 

1 



V 
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notice that 








0^ 











I 


1 









1 


;) 







1 ) 




^ I 








Substitute the above equation into (2.2), by some calculations 
1 



1 

W) 

1 



/3 



/3 



1 

1 1 I MiM2---Mj_i 
1 




1 
1 1 
1 



Ml 




M2 




complete the proof. 



(Pi(0) +P2(0),P2(0))MiM2 ••• M,_ 1(1,0)', 



(1,1,0)' 




:i,l,0)' 



□ 



Proof of Theorem 1.2 



3.1 The Asymptotic Solution of Difference system 

In this section, we introduce the asymptotic behavior of linear difference system. Here, we just 
consider the second-order system. 

yk+i = [K + Rk]yk k>0 (3.1) 

where G M^, A = diag{Ai, A2}, and Rn is a small perturbation in a sense to be made precise. 
We assume that |Ai| > IA2I > 0, and \\Rn\\ = Y17=i Sj=i 

The classical result for asymptotic analysis of solutions is to represent a fundamental matrix 
in the form 

k-i 



Yk =[i+ 0(1)] n 



i=0 



where K{1) is an explicitly diagonal matrix whose main terms come from A. 

If we consider the difference equations yu+i = AkVk- Use this asymptotic representation, we 
can give a precise estimation of the non- homogeneous matrix products AnAn-i ■ ■ ■ Aj-q- 



Here, we just give the case that Rn in (3.1) is an /^-perturbations and /^-perturbations with 



1 < p < 2. If p > 2, the result is more complicate. 



3.2 /^-perturbations 



The fundamental theorem of Levinson (^15j, 1948) establish analogous results for perturbed 
systems of differential equations. Benzaid and Lutz ([Ij, 1987) give the discrete analogue for 
difference equations. This theorem consider the more general case when A is depends on k, 
requiring a dichotomy condition on A and a growth condition on the perturbation R^. 
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Proposition 3.1. (Theorem 2.2 Benzaid and Lutz (P202)) Consider yk+i = [A + Rk]yk, where 
A = diag{Xi, A2 • • • A^}, Aj 7^ for all 1 < i < n, and X^^^-^ ll-^fcll < 00. Then the system has a 
fundamental matrix satisfying, as k ^ 00 

Yk = [I + o{1)]AK (3.2) 
3.3 P-perturbations with 1 < p <2 



While the discrete version of Levinson's theorem considered Z -perturbations R in (3.1), the 
discrete version of the theorem of Hartman-Wintner (j7], 1955)was concerned with P perturba- 
tions for some 1 < p < 2. The proof is based on the so-called Q-transformation which was first 
introduced for differential equations by Harris and Lutz (|6j, 1974) and later on modified for 
difference equations by Benzaid and Lutz ([Ij, 1987). Those methods have been well-established. 

Proposition 3.2. (Corollary 3.4 Benzaid and Lutz (P210)) Consider y^+i = [A + Rk]yk, where 
A = diag{Xi, A2 • • • A„}, |Ai| > IA2I > • • • |A„| > 0, and YlTo ll-^fcP < Z*^*^ some 1 < p < 2. 
Then the system has a fundamental matrix satisfying, as k ^ 00 



k-l 



Yk = [I + o{l)]Y[[A + dmgRi] 



(3.3) 



i=0 



3.4 From "positive recurrence area" to the boundary of null recurrence 

In this section, we just consider when R = 2, and assume that P2ii),P2 > 0. The key to prove 
Theorem 1.2 is to discuss when P{n) — )• P, the asymptotic representation of M1M2 ■ ■ ■ M^. To 
this end, we consider the following difference system. 



yn+i = M'^yn = [M' + 



(3.4) 



where Rn = Mn - M, 
M 



P1+P2 £2 
1 



Pi(n)+p2(") P2{n) 



M„, 



q{n) 
1 



q{n) 





It is easy to see that 1 = Ai > > A2 > —1, where Ai,A2 are two eigenvalues of M. So it 
can be expressed in the diagonal form 



T-^MT = diag(Ai,A2 



where T is the non-singular matrix 



T 



1 A2 
1 1 



Let yn = {T )' Zn, and (3.4) becomes 



Zn+i = [diag(Ai, A2) + T'R'^{T ^)']z„. 



(3.5) 
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Lemma 3.1. Under our condition, we have for some constant Ki,K2, 

KiEn < \\T'R'^{T-^y\\ < K2en, as e„ ^ 0. 

Proof. For fixed T, by P295, Theorem 5.6.7. \\T^^ 'T\\o = || • || is also a norm of the matrix. 
Then by the equivalence of the norm, there exists constants ci, C2, 



We can see 



\Rn\ 



Clll-Rnll < l|-^n||o < C2||-Rn||- 



. Piin)+p2{n) P1+P2 , ^ ^P2{n) P2 



^[n) q ' q{n) q 

Using pi{n) ~ pi — Csn, P2{n) ^ p2 — Csn, q{n) ~ g + Ce„, we have for some constant C", 
||-Rn|| ~ C'sn- So there exist Ki,K2 satisfy 

KiSn < \\T'B!^{T-^)'\\ < K2en 

□ 



Combine (3.4), (3.5), Lemma 3.1 and the discuss above, we deduce from the Propositions 



3.1 and 3.2 that. 



Lemma 3.2. IfYl'n=o^n < oo, Then the system (3.4) has a fundamental matrix satisfying, as 
/c — )• CO 

Yk = {T-^y[I + o{l)]diag{l,X^2)- (3-6) 



Proof. By Lemma 3.1, if X^^q^" ^ Proposition 3.1 to system (3.5), we have 

Zfc = [I + o(l)]diag(l,A^). (3.7) 
Substitute y„ = (T~-^)'z„ we obtain (3.6), complete the proof. □ 



Lemma 3.3. If for some p such that 1 < p < 2, Yl'^=o^^ < '^hen the system (3.4) has a 
fundamental matrix satisfying, as /c oo 



fc-i 



Yk = (r-i)'[/ + o(l)] llidiagT-^MiT] 



(3.8) 



i=0 



Proof. The proof is the same as Lemma 5.2, 

diag(Ai, As) + diagr'<(r-i)' = diag(Ai, A2) + diagr'M;(r-i)' - diagr'M'(r-i)' 

= diagr'M;(r-^)' = diagT- ^M^T 



□ 
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3.5 Proof of Theorem O 



Proof. It is evident that Yi := M[M[_^ ■ ■ ■ M[ is a fundamental matrix of (3.4); on the other 
hand Yi in (3.6) and (3.8) is also the fundamental matrix of (3.4). So there exists a nonsingular 
matrix C such that Yi = Yi ■ C. 
(a) If J2n=0^n < oo. 

Combine Theorem |1.1[ Lemma 3.2 and (3.4), 

1 1 

^ ^ q{i) ^ q{i) ^ * ^ » ^ ^ 

1 



1 



eiYi.iCa' 



ei{T-J[I + o(l)]diag(l, A*2" )Ca' 



Because IA2I < 1, ^(i) has a positive limit as i — )• cxo. So X]i^o/^(^) ~ and X„ is null 
recurrence (It is evident that X„ is recurrent because all the transition probability Pi is in the 
"positive recurrent area D") . 

(b) If Yl'^=o^n = OO) and for some p such that 1 < p < 2, Yl'^=o^^ ^ ^^n calculate 

diagT" iMjT, 



diagr-^M.r 



1 

1 - A2 
1 

1 - Ao 



Pi(n)+2p2(») 



Q(n) 



A2 

\ (-1 _ Pi{n)+2p2{n) \ _ p'zjn) 
q(_n) ' q{n) 

1 - K£n - A2 \ 











P2(n) 
q{n) 



By Lemma 3.3, use the same method, there exist an constant c, such that 



i-l 



(1 - \2)q{i) 



\{{l-Keu). 



k=0 



Using the fact that log(l — a) ~ —a as a — )• we see that 



i-l 



i-l 



log (1 — KEk) ~ — KEi as A; — )• 00, 



ik=0 



A:=0 



then 



Note that 



(1 - A2)g« 



Ei— 1 



(3.9) 



(1-A2)9(i) 



is a bounded sequence. If X^i^o 111=0 ^ (3.9), X]i^o'"(^) ^ 

the process is positive recurrence. If X^^^o 111=0 ^"'^^'^ ~ (3.9), X]£o'"(*) ~ '^^^^ 

recurrence, when the process is positive recurrence. 



log 7r(z) = log 



fc=o 



(3.10) 

□ 
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3.6 Proof of Corollary 



In this section, we consider the example to explain the boundary between null recurrence and 
positive recurrence. For the case -R = 1, it can be found in ([4J). 

Recall that k = ^. We assume pi{0) = Pi — Cn~°', P2{0) = P2 — Cn^°^, and for n > 0, 
Pi{n) = pi — Cn'", P2{n) = P2 — Cn~°', q{n) = q + Cn'", where Pi,P2,Q satisfy pi + 2p2 = q. 



Case 1 : a > 1. We have X^^q^" ^ Theorem 1.2 (a), Xn is null recurrence. 

Case 2 : ^ < a < 1. There exist 1 < p < 2 such that X^^^g ^" ^ ^^'^ 



n 

k=0 



So l^i^onfc=o^ '^^'^ ^ Theorem 1.2 (6), the process is positive recurrence, and we have 

Ck 



log7r(i) 



1 — a 



-i^ as i — >• oo. 



Case 3 : a = 1. It is easy to see that X^^q e„ < oo. And 



n 

fe=0 

So if Ck < 1, E£o nl=o = oo X„ is nuU recurrence; if Ck > 1, E»=o IlLo e""^" < oo 

X„ is positive recurrence, and 

log7r(i) ~ — C/ilogi, as i — )■ oo. 



□ 
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